Applying Perceptually Correct 3D Film Noise

ABSTRACT

Perceptually correct noises simulating a variety of noise patterns or textures may be applied to stereo image pairs each of which comprises a left eye (LE) image and a right eye (RE) image that represent a 3D image. LE and RE images may or may not be noise removed. Depth information of pixels in the LE and RE images may be computed from, or received with, the LE and RE images. Desired noise patterns are modulated onto the 3D image or scene so that the desired noise patterns are perceived to be part of 3D objects or image details, taking into account where the 3D objects or image details are on a z-axis perpendicular to an image rendering screen on which the LE and RE images are rendered.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of priority to related, co-pendingProvisional U.S. Patent Application No. 61/613,338 filed on 20 Mar. 2012entitled “Applying Perceptually Correct 3D Film Noise” by Timo Kunkel etal., hereby incorporated by references in its entirety.

TECHNOLOGY

The present invention relates generally to imaging systems, and inparticular, to imaging systems that process and/or render 3-dimensional(3D) or multi-view images.

BACKGROUND

In general, human eyes perceive 3D images based on the slight(parallactic) disparity of the right eye view and the left eye view by,for example: (i) anaglyph filtering; (ii) linear polarizationseparation; (iii) circular polarization separation; (iv) shutter glassesseparation; (v) spectral separation filtering; (vi) lenticular lensseparation; and parallax barrier screening. The illusion of depth can becreated by providing an image as taken by a left camera in a stereocamera system to the left eye and a slightly different image as taken bya right camera in the stereo camera system to the right eye. Noisesproduced by each of the left and right cameras may comprise desirablenoises such as film grain simulation of chemical films as well asundesirable noises such as digital noises producing noticeable visualartifacts.

A common problem with 3D image capturing is that each of the left andright cameras in the same stereo camera system may have its own distinctfootprint and noise characteristics which typically do not match thoseof its counterpart camera. As a result, the right eye (RE) image and theleft eye (LE) image of a 3D image may comprise noises of differentfootprints and characteristics. Noise reduction techniques may beapplied to the RE and LE images; however, these techniques remove bothdesirable noise and undesirable noise from the RE and LE images. Inaddition, resultant 3D images may become too clean to be perceived asrealistic.

The approaches described in this section are approaches that could bepursued, but not necessarily approaches that have been previouslyconceived or pursued. Therefore, unless otherwise indicated, it shouldnot be assumed that any of the approaches described in this sectionqualify as prior art merely by virtue of their inclusion in thissection. Similarly, issues identified with respect to one or moreapproaches should not assume to have been recognized in any prior art onthe basis of this section, unless otherwise indicated.

BRIEF DESCRIPTION OF DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1A illustrates noise mismatch in a scanline of an example stereoimage pair;

FIG. 1B illustrates an example stereo image pair that may contain a highamount of embedded noise;

FIG. 2 illustrates example depth noises caused by spatial mismatch ofnoise intensities in a stereo image pair;

FIG. 3 illustrates applying a desired noise pattern applied under otherapproaches and under techniques as described herein;

FIG. 4 and FIG. 5 illustrate example adjustments of spatial frequencycomponents of applied noise as a function of depth (z-axis);

FIG. 6A through FIG. 6D illustrate non-limiting example configurationsof an image processing system 600 that applies perceptually correctnoises to 3D images, in accordance with an embodiment of the presentinvention;

FIG. 7 illustrates an example process flow; and

FIG. 8 illustrates an example hardware platform on which a computer or acomputing device as described herein may be implemented, according apossible embodiment of the present invention.

DESCRIPTION OF EXAMPLE POSSIBLE EMBODIMENTS

Example possible embodiments, which relate to applying perceptuallycorrect noises, are described herein. In the following description, forthe purposes of explanation, numerous specific details are set forth inorder to provide a thorough understanding of the present invention. Itwill be apparent, however, that the present invention may be practicedwithout these specific details. In other instances, well-knownstructures and devices are not described in exhaustive detail, in orderto avoid unnecessarily obscuring the present invention.

Example embodiments are described herein according to the followingoutline (outline section headings are for reference purposes only andshall not in any way control the scope of the present invention):

-   -   1. GENERAL OVERVIEW    -   2. 3D IMAGES    -   3. PERCEPTUALLY INCORRECT NOISE    -   4. PERCEPTUALLY CORRECT NOISE    -   5. NOISE APPLICATION    -   6. VARYING SPATIAL RESOLUTIONS OF APPLIED NOISE BASED ON DEPTHS    -   7. SYSTEM CONFIGURATIONS    -   8. PROCESS FLOW    -   9. IMPLEMENTATION MECHANISMS—HARDWARE OVERVIEW    -   10. EQUIVALENTS, EXTENSIONS, ALTERNATIVES AND MISCELLANEOUS

1. GENERAL OVERVIEW

This overview presents a basic description of some aspects of a possibleembodiment of the present invention. It should be noted that thisoverview is not an extensive or exhaustive summary of aspects of thepossible embodiment. Moreover, it should be noted that this overview isnot intended to be understood as identifying any particularlysignificant aspects or elements of the possible embodiment, nor asdelineating any scope of the possible embodiment in particular, nor theinvention in general. This overview merely presents some concepts thatrelate to the example possible embodiment in a condensed and simplifiedformat, and should be understood as merely a conceptual prelude to amore detailed description of example possible embodiments that followsbelow.

In some embodiments, stereo image pairs processed under techniques asdescribed herein may comprise left eye (LE) images and right eye (RE)images, for example, captured with two different cameras in a multi-viewor stereoscopic camera system as analog or digital images. Due todifferences in noise-related characteristics of cameras and otherinfluencing factors in image acquisition, noises generated in LE imageand the RE image of a stereo image pair are perceptually incorrect, evenif the LE and RE images are taken simultaneously or substantiallysimultaneously (e.g., with a fraction of a cent-second, millisecond,microsecond, etc.). As used herein, “noises in the LE image and the REimage being perceptually incorrect” means that those noises are randomlycorrelated spatially, or incorrectly correlated spatially. For example,noises introduced in image acquisition may appear as a mist of dropletsat or near a plane of a particular depth in a z-axis (which may beperpendicular to an image rendering screen on which the LE image and theRE image are rendered).

In some embodiments, perceptually incorrect noises embedded in (input)LE and RE images are first removed from the LE and RE images. Desirednoise patterns may then be applied to the LE and RE images. As usedherein, “desired noise patterns or prints” refer to system configuredand/or user selected noise patterns or prints that are to be applied toLE and RE images in a perceptually correct manner that takes intoaccount the underlying depth information of 3D image content or detailsin an 3D image represented by the LE and RE images.

In some embodiments, desired noise patterns are modulated onto the 3Dimage or scene so that the desired noise patterns are perceived to bepart of 3D objects or image details, taking into account where the 3Dobjects or image details are on the z-axis. Under techniques asdescribed herein, the desired noise patterns as applied to the LE imageand the desired noise patterns as applied to the RE image (1) arerelated to, or derived from, the same system-configured or user-selecteddesired noise patterns, and (2) are offset between each other with thesame depth information (e.g., disparity map, metadata, computeddisparity or the like) as that of the 3D objects or image details towhich the desired noise patterns are applied. As used herein, a 3Dobject or image detail refers to a visible/perceptible portion of 3Dcontent in a 3D image represented by one or more corresponding pixels inLE and RE images of a stereo image pair.

In some embodiments, the spatial resolutions of the desired noisepatterns may be adjusted based on depth values of 3D objects or imagedetails. For example, to improve the perceptual accuracy, the spatialresolution of a desired noise pattern may be adjusted higher for objectsthat are close to a viewer and lower for other objects that are furtheraway from the viewer. In some embodiments, this may be implemented byshifting spatial frequencies of the desired noise pattern to highervalues or by increasing high spatial frequency components (or decreasinglow spatial frequency components) in the desired noise pattern for theobjects that are close to the viewer, and by shifting spatialfrequencies of the desired noise pattern to lower values or bydecreasing high spatial frequency components (or increasing low spatialfrequency components) in the desired noise pattern for the objects thatare farther away from the viewer.

In some embodiments, adjustments of spatial resolutions of desired noisepatterns based on depth values of 3D objects or image details do nothave to follow a linear relationship. Instead of using a linearrelationship, a non-linear relationship may be used to scale the spatialresolution of a desired noise pattern with the depth values. Under somelinear or non-linear relationships, a particular depth value range(e.g., one or more of front, middle, or back regions of the 3D scene)may be overemphasized with even higher spatial frequencies or even morehigh spatial frequency components, relative to some other linear ornon-linear relationships. This may be used to perceive some depthsbetter or to perceive some other depths less for instance leading tocomplete or partial compression and/or expansion of the depth axis.Overemphasizing with high spatial frequencies and underemphasizing withlow spatial frequencies may be used in noise applications whether or notdisparity or displacement values for the distance between a coplanarplane and the perceived position of 3D objects are correspondinglyincreased or decreased using the same relationships for with highspatial frequencies and underemphasizing with low spatial frequencies.

In some embodiments, mechanisms as described herein form a part of animage processing system, including but not limited to: a server, studiosystem, art director system, image editor, color grading or masteringtool, professional reference monitor, animation system, movie studiosystem, cameras, TVs, broadcast system, media recording device, mediaplaying device, video projector, screen (e.g., matte screen, grayscreen, silver lenticular screen or the like), laptop computer, netbookcomputer, tablet computer, cellular radiotelephone, electronic bookreader, point of sale terminal, desktop computer, computer workstation,computer kiosk, or various other kinds of terminals and display units.

Various modifications to the preferred embodiments and the genericprinciples and features described herein will be readily apparent tothose skilled in the art. Thus, the disclosure is not intended to belimited to the embodiments shown, but is to be accorded the widest scopeconsistent with the principles and features described herein.

2. 3D IMAGES

A 3D image as described herein may be represented by a stereo image pairthat comprises an LE image and an RE image. The LE image comprises (LE)pixels that are to be viewed by a viewer's left eye, whereas the REimage comprises (RE) pixels that are to be viewed by a viewer's righteye.

Image features represented in a 3D image may be located at differentdepths along a z-axis vertical to a coplanar view plane in the 3D image.As used herein, the term “coplanar plane” or “coplanar view plane”refers to an imaginary plane in the 3D image (or scene); LE pixelsassociated with (e.g., portraying or representing) a feature on thecoplanar view plane have zero disparity with corresponding RE pixelsassociated with the same feature on the coplanar view plan. In contrast,LE pixels associated with features not on the coplanar view plane arehorizontally displaced with respect to RE pixels associated with thesame features.

A horizontal displacement between a LE pixel and a RE pixel,corresponding to the same point or location of a feature in the 3Dimage, is represented or characterized by a per-pixel disparity. Valuesof per-pixel disparity for different pairs of corresponding LE and REpixels may vary with depths, along the z-axis, of the featuresrepresented by the different pairs of LE and RE pixels.

If disparity information between LE pixels and RE pixels is correct,then a viewer is able to correctly perceive the positions, shapes,relative spatial relations of the features represented in the 3D image.On the other hand, if disparity information between LE pixels and REpixels is incorrect or does not exist, then a viewer is not able to doso.

3. PERCEPTUALLY INCORRECT NOISE

FIG. 1A illustrates noise mismatch in a scanline of an example stereoimage pair comprising an LE image (frame) and an RE image (frame). Theexample stereo image pair may be rendered on a display panel or imagerendering surface (e.g., in a frame sequence) and perceived by a vieweras a single 3D image as illustrated in FIG. 1B. Example stereo imagepair of FIG. 1B may contain a high amount of embedded noise (e.g., imagesensor noise, etc.). The scanline of FIG. 1A may be based upon thepixels from the scanline as illustrated in FIG. 1B.

In some embodiments, pixels in the example scanline from one or both ofthe LE and RE images comprise a high level of noises (e.g., digitalsensor noises, etc.) embedded in the LE and RE images. Examples ofembedded noises may be, but not limited to any of: intrinsic noises,device specific noises, processing induced noises, compression noises,etc. The noises in the LE and RE images may be perceptually incorrectand may not be correlated with correct disparity information. Forexample, as illustrated in FIG. 1A, while the stereo image pair isaligned as a viewer perceives, the noise embedded in the LE and REimages do not spatially match or correlate between left and right image.This spatial mismatch of the noise intensities causes errors indisparity information or depth information for pixels affected by thenoises, and thus depth noises in the 3D image.

FIG. 2 illustrates example depth noises caused by spatial mismatch ofnoise intensities in an example stereo image pair as illustrated in FIG.1A and FIG. 1B. As shown, the LE and RE images may comprise correctdisparity information that allows a viewer to perceive a figure holdinga flower branch with correct depth information of the features of thefigure and the flower branch and their relative spatial relationship.However, the LE and RE images comprise visible noises (202) that are notrelated with correct depth information or disparity information. Thevisible noises (202) may represent a magnified noise pattern from anactual 3D image. These visible noises (202) may be perceived asunrelated to, or away from, the figure and the flower. While the figureand the flower branch may be around a particular viewing plane with aparticular depth in the z-axis, the visible noises (202) over the figureand the flower branch may appear away from the figure and the flowerbranch, and may be around a different viewing plane with a differentdepth in the z-axis. The visible noises (202) may be centered at or nearthe coplanar viewing plane or an accommodation plane to which a viewer'sperception tends to accommodate or accustom. As a result, these noises(202) do not convey desired noise patterns such as film grain withcorrect depth information in any correlation with the objects or figuresin the 3D image; instead, a viewer of the 3D image may perceive thesenoises (202) as some visible extraneous matter floating in front of, orbehind, the portrayed figure and flower branch. While 3D objects orimages may be into or out of the screen in a 3D image, the noises maynevertheless still appear at the screen or at a depth at which theviewer's perception tends to accommodate as mist of droplets. As aresult, these noises (202) may be very distracting to the viewer and mayseverely confuse the human visual system (HVS) leading to nausea andheadaches.

4. PERCEPTUALLY CORRECT NOISE

In some embodiments, noises embedded in a stereo image pair may be firstremoved and then a desired noise pattern may be applied to both LE andRE images in the stereo image pair.

Applied noises as described herein may refer to any kind of perceptiblepattern or texture including film grains (e.g., simulating those of ISO1600 chemical films), watermarks (which, for example, indicates that theimages are provided for previewing purposes only), etc.; here, theapplied noises injected into an LE and RE images of a stereo image pairis correlated. A noise portion in the applied noises may be applied toan image feature (or a portion thereof) at a certain z-value or depth(e.g., slightly, perceptibly, etc.) to make the image feature (or theportion thereof) go up and down, left and right, or in and outspatially. Additionally, optionally, or alternatively, a noise portionin the applied noises may alter luminance of an image feature (or aportion thereof) at a certain z-value or depth from bright to dark, fromdark to bright. Additionally, optionally, or alternatively, a noiseportion in the applied noises may vary the color of an image feature (ora portion thereof) at a certain z-value or depth. Such a noise portionmay be applied to the LE image and the RE image in the form of twocorrelated noise portions that are derived or built from the formernoise portion with appropriate displacement or depth information inbetween the correlated noise portions.

Since clean images without noises may be perceived as unreal or surreal,applied noises as described herein may also be used to make images to beperceived as realistic images. Applied noises as described herein maysimulate a variety of response curves, saturation properties, noiseproperties, etc., as related to one or more of a variety of media ordevices. The distribution properties of applied noises as describedherein may include, but are not limited to, uniform distribution,regular distribution, random distribution, Gaussian distribution,distribution patterns of one or more of other types of probabilitydensity functions (PDFs), etc.

In some embodiments, applied noises are specified with transferfunctions; to apply the noises, existing or noise-removed pixel valuesin a stereo image pair of an LE image and an RE image may be transformedwith one or more of the transfer functions. The transfer functions asapplied to the LE image and the RE image may be spatially modified,respectively for the LE image and the RE image, by displacement ordisparity information relating to depth values of 3D image objects ontowhich the noises are applied.

Applied noises as described herein may comprise intensity variations,color variations, halos, and other possible color or luminance noiseeffects. In some embodiments, noises may be applied to bright areas(e.g., on 3D objects or image features) differently from how noises maybe applied to dark areas in order to simulate analogous chemicalproperties of silver, color layers, etc.

Applied noises as described herein may also be generated based onembedded noises or noise properties thereof. In an example, input imagedata (e.g., a 3D image sequence comprising a sequence of stereo imagepairs) may comprise image metadata, in addition to image data samples;the image metadata may specify/suggest a particular type of noise to beapplied to a particular 3D image or frames. In another example, existingnon-perceptual 2D noises may be removed and analyzed to determine aparticular noise pattern to be combined with depth information for thepurpose of applying the particular noise pattern as perceptually correct3D noises. Thus, techniques as described herein may be used to convertnon-perceptual sensor noises or device specific noises into perceptuallycorrect noises in 3D images or scenes. These techniques may be used, forexample, to simulate sensor noises or specific looks of a specificdevice. Noise application as described herein may use one or more ofsampling techniques, modeling techniques, techniques based on matrix,textural patterns, computer-generated graphics, wood textures, othermaterial textures, etc. Applied noises as described herein also may beprovided at least in part by an external noise source to an imageprocessing system (e.g., 600 of FIG. 6C), instead of generating whollyby the image processing system.

As noted above, in some embodiments, embedded noises are first removedor reduced in an input stereo image pair. However, in some otherembodiments, this noise removing step may be skipped (e.g., with aninput stereo image pair comprising clean images with little or noembedded noises, or in a scenario in which embedded noises in inputimages are not removed).

Images with noises may consume more bandwidth than clean images. Thus,in some embodiments, clean images are provided to an image processingsystem such as a TV; the image processing system may apply desired noisepatterns or prints as appropriate. The TV may store a plurality of noisepatterns and may select one or more of the stored patterns based onimage metadata. The TV may be configured to analyze input images andcompute depth information based on the results of analyzing the inputimages. Additionally, optionally, or alternatively, depth informationmay be given as a part of image metadata to the TV, which may apply thereceived depth information with a particular noise pattern or printwithout analyzing the input images for the purpose of computing thedepth information thereof. Additionally, optionally, or alternatively,the global spatial distribution provided e.g. by metadata can be scaled,distorted, skewed or otherwise manipulated as a function of theobservers distance to the display unit by using methods such ashead-tracking, eye-tracking, laser scanning, depth imaging, etc. Thisensures that the noise always has the same retinal spatial frequenciesindependent of the observer's distance to the display.

5. NOISE APPLICATION

Noise application may be performed for the whole image sequence, forwhole images, for parts of images, for a feature film but not to acommercial break, etc. For example, a desired noise pattern or print maybe applied to a particular 3D object or figure (e.g., a weatherforecaster) but not to other image details surrounding the particular 3Dobject or figure. In a further example, the particular 3D object orfigure may also be used for image superimposition purposes in 3Dvideo/image applications. In some embodiments, a user may provide userinput (through a remote, input device, keypad, etc.) to specify whethermedia content should be treated with perceptually correct noises asdescribed herein. Additionally, optionally, or alternatively, the usermay also provide user input to specify various properties (e.g., amount,one of multiple noise patterns, etc.) of the noises to be applied to themedia content.

Techniques as described herein may be used to apply desired noisepatterns or prints to images related to virtual reality, 3D mappingapplications, captured at film planes by image sensors, etc., so long asdepth information (which may include but is not limited to volumetric orstereoscopic information) related to these images may be determined.Techniques as described herein may be used in color grading processing.

FIG. 3 illustrates a desired noise pattern applied under otherapproaches and a desired noise pattern applied under an approach thatuses techniques as described herein. For the purpose of illustrationonly, in FIG. 3, a solid contour with two wedge shapes represents depthinformation of image features portrayed in the 3D image, while rapidlychanging wave forms represent applied noises overlaid onto the 3D imageunder the other techniques and under the techniques as described herein.The image features in the 3D image or scene may be located at differentdepths. As illustrated, object 1 may be behind an accommodation plane;object 2 may be in front of the accommodate plane; other image featuresmay or may not be located at the accommodation plane.

As used herein, the accommodate plane may refer to a plane at whichnoises are applied under other approaches that do not use techniques asdescribed herein. Examples of the accommodation plane may include butare not limited to a coplanar plane, a plane corresponding to an imagerendering screen (e.g., projection screen) as illustrated in FIG. 3, aplane at a depth where a viewer's vision tends to accommodate, etc.

Under other approaches, applied noise (302) appears to lie at or nearone or more specific planes such as the accommodation plane of FIG. 3,regardless of where image details may be located. As illustrated in FIG.3, for image details located at the accommodation plane, noises (302)are applied at the accommodation plane. Similarly, for image details(e.g., object 1 and object 2) not located at the accommodation plane,noises (302) are still applied at the same accommodation plane. Thus,the noises (302) may appear floating away from the image details (object1 and object 2). Applied noises (302) may appear in front of object 1but behind object 2. For the same reasons as discussed with noisesembedded within the stereo image pair, the applied noise (302) under theother approaches is prone to cause perceptual confusion for the HVS.

In contrast, under an approach adopting techniques as described herein,applied noise (304) may be modulated with 3D depth information specificto the 3D image or scene represented by the stereo image pair. Forexample, the applied noise (304) may be modulated with depth informationto cause the applied noise (304) to be on top of, coincide, orsubstantially coincide, with 3D elements or image features in the 3Dimage or scene. As illustrated in FIG. 3, applied noises (304) areco-located with the image details whether the image features are at theaccommodation plane, or not at the accommodation plane. Specifically,for noises (304) applied to object 1 and object 2, they are set with thesame depth information with the image features corresponding to object 1and object 2. As a result, extraneous visible artifacts caused by depthnoises are avoided under techniques as described herein

In some embodiments, applied noises are created with, or represented by,one or more noise patterns. In an example, the accommodation plane maybe a coplanar view plane where LE pixels and corresponding RE pixelsportraying image features at the coplanar view plane have zero disparityvalues. Other LE pixels and other corresponding RE pixels portrayingimage features at a different depth from that of the coplanar view planemay have non-parity values.

In some embodiments, disparity values of pixels in an LE or RE image (orframe) are expressed as offsets to corresponding pixels in a referenceimage. As used herein, pixels from images of different views (LE, RE, orreference) correspond to one another if the pixels represent the sameimage features in a 3D image or scene. In some embodiments, one of theLE or RE images may be set as the reference image; thus, pixels in thereference image have zero offsets; some pixels in the other (e.g.,non-reference image, etc.) of the LE or RE images have zero offsets ifthe pixels represent image features in the coplanar view plane; and someother pixels in the non-reference image have non-zero offset if thepixels represent image features not in the coplanar view plane. In someembodiments, an image with a different view (e.g., center view, anintermediate view between the views associated with the LE or RE images,etc.) may be set as the reference image; thus, some pixels in the LE andRE images have zero offsets if the pixels represent image features inthe coplanar view plane; and some other pixels in the LE and RE imageshave non-zero offset if the pixels represent image features not in thecoplanar view plane.

Under techniques as described herein, a noise pattern applied to the 3Dimage or scene is adjusted with appropriate offsets in the LE and REimages so as to perceptually lie on top of 3D objects and image featureswhich may or may not be located at the accommodation depth (e.g., thedepth of the accommodation plane). In some embodiments, the noisepattern may be applied to a reference image. The noise pattern in thereference image is then adjusted with the same offsets as the underlyingpixels.

In some embodiments, varying offsets of different noise portions of thenoise pattern in the LE and RE images perceptually move these differentnoise portions to different positions on the z-axis. In someembodiments, these different noise portions match the positions on thez-axis of 3D objects or image details so as to perceptually lie on the3D objects or image details.

For example, if a pixel in the reference image corresponds to a pixel inan LE or RE image with a certain offset, then a noise portion (e.g., aportion of the noise pattern) applied to the pixel in the referenceimage is adjusted with the same offset so as to perceptually lie on topof the corresponding pixel in the LE or RE image.

In some embodiments, only noise portions that applied to image featuresat the accommodation depth are given zero offset; thus, these noiseportions in the LE image match the same noise portions in the RE image.On the other hand, other noise portions that applied to image featuresnot at the accommodation depth are given non-zero offsets innon-reference image(s); thus, these noise portions in the LE image donot match the same noise portions in the RE image.

6. VARYING SPATIAL RESOLUTIONS OF APPLIED NOISE BASED ON DEPTHS

Applied noises as described herein may comprise a set of spatialfrequency components at different spatial frequencies, which are in afrequency domain transformed from a spatial domain comprising values inx and y axis of the reference image. Here, the x and y axis may refer topixel columns and rows on a display panel or image rendering screen. Insome embodiments, the magnitudes of the spatial frequency components inthe applied noises may vary as a function of depth. This function may bepiece-wise, linear, non-linear, analytical, non-analytical,table-driven, etc. In some embodiments, if an object is closer to anobserver, the spatial frequencies of the applied noise may be sethigher. Similarly, if an object is farther away from the observer, thespatial frequencies may be set lower.

FIG. 4 illustrates example adjustments of spatial frequency componentsof applied noise as a function of depth (z-axis). As illustrated, thecloser a 3D object or image detail is to the viewer in the scene, thehigher the spatial frequency of the applied noise. This may beaccomplished by increasing or shifting spatial frequencies of thespatial frequency components and/or by increasing weights of highspatial frequency ones of the spatial frequency components. On the otherhand, the farther away a 3D object or image detail is to the viewer inthe scene, the lower the spatial frequency of the applied noise. Thismay be accomplished by decreasing or shifting spatial frequencies of thespatial frequency components and/or by decreasing weights of highspatial frequency ones of the spatial frequency components.

FIG. 5 shows an example simulation of perceptual noise using spatialfrequency adjustments as a function of depth. The top left image shows adepth map of the image while the top right image gives the depthadjusted noise pattern. It is visible that scene elements closer to theobserver have a higher spatial frequency than those further away. Thelower image in FIG. 5 finally shows the noise print combined with theoriginal scene.

In a further embodiment, high spatial frequency components in appliednoises may be overemphasized or underemphasized to cause visualperceptions of increased or decreased perceived depths. For example,instead of linearly scaling spatial frequency components in the appliednoises, spatial frequency components in the applied noises may benon-linearly scaled to overemphasize high spatial frequency componentsto cause a visual perception of a steeper or exaggerated change ofspatial resolution thus a visual perception of larger depth (in thez-axis) than is stereoscopically visible. This technique may be used tohelp alleviate headaches and nausea caused by the disparity between aperceived depth and the actual accommodation point (e.g., the imagerendering screen).

In some embodiments, displacement/disparity (in horizontal planeperpendicular to the z-axis) of corresponding pixels in twocorresponding LE and RE images may be linearly proportional to the depthof a location represented by the corresponding pixels. Human eyes mayaccommodate to perceive objects at an accommodation plane, which may bebut is not limited to, a display screen, display eyeglasses, etc.Scaling displacement of pixels linearly with depth may cause objects tobe perceived as jumping too closely to the viewer, causing an uneasyfeeling in the viewer. Similarly, objects away from the accommodationplane may be perceived as fading away too quickly.

In some embodiments, in order to alleviate or avoid thesepsycho-perceptual issues, displacement/disparity of corresponding pixelsmay be scaled nonlinearly with the actual depth of the point representedby the corresponding pixels. In some embodiments, limit functions thatnever actually reach certain maximum limits may be used to relate thedisplacement/disparity with the depth. In some embodiments, z values maybe compressed (e.g., the depth perspective may be foreshortened into aninterval that allows the viewer to have a perceptual feeling of being asafe observer). In some embodiments, z values may be expanded (e.g., forthe purpose of deliberately causing a particular viewer reaction).

In some embodiments, information about a particular viewing environmentmay be received by an image processing system as described herein. Theviewing environment information may be used to relate depth informationwith displacement/disparity information. For example, a head-trackingmechanism may be deployed within or in conjunction with the imageprocessing system to determine geometric relationships of a viewerrelative to an image rendering screen. These geometric relationships mayinclude but are not limited to the viewing distance, the viewing angle,etc. These geometric relationships may be used in determining howdisplacement/disparity of corresponding pixels is related to a depth ofthe point represented by the corresponding pixels.

Under techniques as described, desired noise patterns or prints may beapplied in various embodiments in which displacement/disparity may berelated to the depth in any one or more of a wide variety of differentways, including but not limited only to, those as discussed above.

7. SYSTEM CONFIGURATIONS

As discussed herein, noises embedded in LE and RE images of a stereoimage pair (e.g., related to a real footage) such as illustrated in FIG.1A and FIG. 1B may not spatially or perceptually match between the LEimage and the RE image. As noted, in some embodiments, embedded noisesin LE and RE images in a stereo image pair may be first removed orreduced. In some embodiments, embedded noises may include but are notlimited only to, noises perceptually incorrectly applied by imageprocessing or acquisition devices. In some embodiments, previouslyapplied noises may also be removed or reduced under the techniques asdescribed herein.

In some embodiments, a desired noise pattern or print may becomputationally applied or reapplied to the stereo image pair. Thedesired noise pattern or print may include but is not limited only toone creating a classic film stock grain. LE and RE noise patternsrespectively applied to the LE and RE images may be derived from thesame desired noise pattern. For example, the LE noise pattern applied tothe LE image may be adjusted with the same offsets as the underlyingpixels of the LE image relative to corresponding pixels in a referenceimage or alternatively in the RE image. Similarly, the RE noise patternapplied to the RE image may be adjusted with the same offsets as theunderlying pixels of the RE image relative to corresponding pixels inthe reference image or the LE image (e.g., when the LE image is selectedas the reference image).

FIG. 6A illustrates an example image processing system 600 thatimplements at least some of the techniques as described herein, inaccordance with an embodiment. In an embodiment, the image processingsystem (600) generally represents a single device or multiple devicesthat are configured to process 3D images. In an embodiment, asillustrated in FIG. 6A, the image processing system (600) may comprise anoise application unit (602), a depth information unit (606), and anoise pattern generation unit (608).

In an embodiment, the image processing system 600 is configured toreceive a stereo image comprising an LE image (604-L) and an RE image(604-R).

In an embodiment, the depth information unit (606) generally representsany hardware and/or software configured to receive two or more inputimages or frames representing different perspectives/views of a 3D imageor scene to produce per-pixel displacements between two different images(e.g., LE image 604-L and RE image 604-R) of a stereo image pair asdepth information (612) and to provide the depth information (612) toother units or devices (e.g., noise application unit 602). In someembodiments, the depth information (612) may be provided by the depthinformation unit (606) in the form of a disparity map. In an embodiment,the depth information (612) produced by a depth information unit (606)as described herein may include both per-pixel displacements, referenceimage information, other disparity-related information, or other imagefeature information constructed based on the per-pixel displacements.

In an embodiment, the noise pattern generation unit (608) corresponds toany device configured to generate one or more noise patterns (610) to beapplied to a 3D image or scene represented by the LE image (604-L) andthe RE image (604-R) and to provide the noise patterns (610) to otherunits or devices (e.g., noise application unit 602). In someembodiments, the noise patterns (612) may comprise a plurality ofapplied noise portions to be applied to individual pixels or individualpixel groups. In some embodiments, the image processing system (600), orthe noise pattern generation unit (608), is configured to accept a userinput (e.g., through a button on the image processing system 600,through a remote operably linked with the image processing system 600,or through a user interface provided by the image processing system 600)that specifies which one or more noise patterns are to be applied; thenoise pattern generation unit (608) is configured to generate, based onthe user input, the one or more noise patterns (610).

In an embodiment, the noise application unit (602) corresponds to anydevice configured to receive stereo image pairs (one of which may be thestereo image pair comprising the LE image 604-L and the RE image 604-R),depth information (e.g., the depth information 612 associated with ordetermined from the LE image 604-L and the RE image 604-R), and appliednoise information (e.g., the noise patterns 610 to be applied to the LEand RE images in the stereo image pair). The noise application unit(602) may be further configured to apply the noise patterns (610), basedat least in part on the depth information (612), to the LE and RE images(604-L and 604-R) to generate a noise applied LE image (614-L) and anoise applied RE image (614-R).

In some embodiments, the noise application unit (602) may first applythe noise patterns (610) to one (e.g., the LE image 604-L) of the LE andRE images (604-L and 604-R), without adjusting the noise patterns (610)using the depth information (612) to generate a corresponding noiseapplied image (the noise applied LE image 614-L in the present example).The noise application unit (602) may generate adjusted noise patternsbased on the noise patterns (610) and the depth information (612). Theadjusted noise patterns may be obtained by adding offsets to noisepatterns (61) using the per-pixel disparity information in the depthinformation (612). The noise application unit (602) may be configured toapply the adjusted noise patterns to the other (the RE image 604-R inthe present example) of the LE and RE images (604-L and 604-R) togenerate another corresponding noise applied image (the noise applied REimage 614-R in the present example). The offsets may be so chosen thatwhen the noise patterns (612) are viewed (with the noise applied LEimage 614-L in the present example) in a first perspective (left eye inthe present example) and the adjusted noise patterns are viewed (withthe noise applied RE image 614-R in the present example) in a seconddifferent perspective (right eye in the present example) resultant 3Dnoise patterns thus perceived match the 3D positions of 3D objects orimage features in the 3D image or scene.

In some embodiments, the noise application unit (602) may first applythe noise patterns (610) to a reference image without adjusting thenoise patterns (610) using the depth information (612). The referenceimage may represent a different view of the 3D image or scene and may beneither of the LE and RE images (604-L and 604-R). In some embodiments,necessary information for constructing the reference image may beincluded in the depth information (612). The noise application unit(602) may generate adjusted noise patterns based on the noise patterns(610) and the depth information (612) for the LE and RE images (604-Land 604-R), respectively. The adjusted noise patterns for each of the LEand RE images (604-L and 604-R) may be obtained by adding offsets tonoise patterns (61) using the per-pixel disparity information in thedepth information (612) for the each of the LE and RE images (604-L and604-R). The noise application unit (602) may be configured to apply theadjusted noise patterns to each of the LE and RE images (604-L and604-R) to generate each of the corresponding noise applied images (614-Land 614-R). The offsets may be so chosen that when the adjusted noisepatterns are viewed (with the noise applied LE and RE images 614-L and614-R) in their respective perspectives, resultant 3D noise patternsthus perceived match the 3D positions of 3D objects or image features inthe 3D image or scene.

In an embodiment, the noise application unit (602) may be furtherconfigured to vary or adjust the mix of spatial frequency components inthe applied noises based on a relationship to the depth in the z-axis.For example, the noise application unit (602) may shift spatialfrequencies of the applied noises to higher values or increasemagnitudes of high spatial frequency components in the applied noisesrelative to those of low spatial frequency components in the appliednoises, dependent on the relationship to the depth in the z-axis. Therelationship to the depth in the z-axis may comprise one or morez-dependent relationships that are linear, non-linear, monotonicallyincreasing, monotonically decreasing, non-monotonic, continuous,algebraic, non-algebraic, etc.

In some embodiments, the image processing system (600), or the noiseapplication unit (602), is configured to accept a user input (e.g.,through a button on the image processing system 600, through a remoteoperably linked with the image processing system 600, through a userinterface provided by the image processing system 600, etc.) thatspecifies which relationship is to be used in scaling spatial frequencycomponents in the applied noises; the noise application unit (602) isconfigured to scale or adjust spatial frequencies or the mix of spatialfrequency components in applying the desired noise patterns or prints tothe stereo image pair.

In some embodiments, a particular section (front, center, back, or asalient part of the 3D scene, etc.) of the z-axis may be overemphasizedor underemphasized with relatively high or low spatial frequencycomponents in the applied noises in relation to other sections of thez-axis.

In some embodiments, applied noises may be applied to entire frames ofLE and RE images forming a stereo image pair. For example, a noisepattern or print may be applied to all 3D image features or objects in a3D image. In some embodiments, applied noises may be applied to only oneor more parts, but not all, of the entire frames of such LE and REimages. For example, a noise pattern or print may be applied to aspecific 3D image feature or object in a 3D image.

For the purpose of illustration only, it has been illustrated that adepth information unit (606) in an image processing system (600) produceper-pixel displacements between two different images (e.g., LE image604-L and RE image 604-R) of a stereo image pair as depth information(612). It should be noted that various other ways may be additionally,optionally, or alternatively, used by the image processing system (600)to determine depth information (612).

FIG. 6B illustrates an alternative system configuration of the imageprocessing system (600), in accordance with an embodiment. As shown, thedepth information (612) may be provided as a part of input image dataalong with an input stereo image pair comprising an LE image (604-L) andan RE image (604-R). For example, the depth information (612) may beprovided as a part of image metadata in the form of a disparity mapcomprising depth information needed for applying desirable noisepatterns or prints.

For the purpose of illustration only, it has been illustrated that adepth information unit (608) in an image processing system (600) produceone or more noise patterns (610) to be applied to a stereo image pair.It should be noted that various other ways may be additionally,optionally, or alternatively, used by the image processing system (600)to determine the one or more noise patterns (610).

FIG. 6C illustrates an alternative system configuration of the imageprocessing system (600), in accordance with an embodiment. As shown, theone or more noise patterns (610) may be determined from input image datacomprising an input stereo image pair of an LE image (604-L) and an REimage (604-R). In an example, the noise patterns (610) may be providedas a part of image metadata in the input image data. In another example,the noise patterns may be decoded or determined from at least a part ofnoises embedded with the LE image (604-L) and the RE image (604-R).

For the purpose of illustration only, it has been illustrated that animage processing system (600) receives, as input, a stereo image paircomprising an LE image (604-L) and an RE image (604-R). It should benoted that various other ways may be additionally, optionally, oralternatively, used by the image processing system (600) to receive oracquire the LE image (604-L) and the RE image (604-R).

FIG. 6D illustrates an alternative system configuration of the imageprocessing system (600), in accordance with an embodiment. As shown, theimage processing system (600) may comprise a camera unit comprising aleft camera (616-L) and a right camera (616-R), which may be configuredto acquire the LE image (604-L) or the RE image (604-R) from reality.

In an embodiment, the left camera (616-L) or the right camera (616-R)corresponds to any device configured to acquire images in terms of fieldraw source frames As used herein, “field raw source frames” may refer toa version of images captured from a reality in image planes (or filmplanes) of image acquisition devices present in the reality; the fieldraw source frames (or simply source frames) may be, but not limited to,a high-quality version of original images that portray the reality.However, in various embodiments, source frames may generally refer to aversion of initial frames that are to be edited, upsampled, downsampled,and/or compressed, along with possible metadata, into input images toother image processing units or devices. In some embodiments, the fieldraw source frames may include artificially created, artificiallyenhanced, or synthesized image frames. In the present example, thesource frames may be captured from a camera system with a high samplingrate that is typically used by a professional, an art studio, abroadcast company, a high-end media production entity, etc. In variousembodiments, the camera unit may comprise two or more image acquisitiondevices each of which may be a single camera element (e.g., the leftcamera 616-L or the right camera 616-R) configured to capture a specificview of a reality. Example of image acquisition devices may include, butare not limited to, a left camera (616-L) and a right camera (616-R) asillustrated, where the camera unit may be a stereo camera system. Insome embodiments, disparity and/or depth information and/or parallacticinformation may be generated by the camera unit comprising the camera(616-L) and the right camera (616-R) and provided to other imageprocessing units or devices. In some embodiments, the camera unit may beconfigured to acquire and communicate geometry information related tooptical configurations of the image acquisition devices to other partsof the image processing system (100). Examples of geometry information(120) may include, but are not limited to, information related topositions and/or offsets of principal points and parallaxes in opticalconfigurations of the image acquisition devices as functions of time.

8. PROCESS FLOW

FIG. 7 illustrates an example process flow according to an exampleembodiment of the present invention. In some example embodiments, one ormore computing devices or components may perform this process flow. Inblock 710, an image processing system (e.g., 600 as illustrated in FIG.6A through FIG. 6D) receives a left eye (LE) image and a right eye (RE)image that represent a 3D image.

In block 720, the image processing system (600) determines, based ondepth information relating to the LE and RE image, one or more firstdepths of a first 3D image feature in the 3D image.

In block 720, the image processing system (600) applies one or morenoise patterns to the first 3D image feature in the 3D image based onthe one or more first depths of the first 3D image feature in the 3Dimage.

In some embodiments, the image processing system (600) may be furtherconfigured to perform: determining, based on the depth informationrelating to the LE and RE image, one or more second depths of a second3D image feature in the 3D image; and applying the one or more noisepatterns to the second 3D image feature in the 3D image based on the oneor more second depths of the second 3D image feature in the 3D image;wherein the one or more noise patterns are applied to the first 3D imagefeature with one or more first spatial frequency components that aredifferent from one or more second spatial frequency components withwhich the one or more noise patterns are applied to the second 3D imagefeature.

In some embodiments, the first 3D image feature is closer to a viewer ofthe 3D image than the second 3D image feature; and the one or more firstspatial frequency components comprise more high spatial frequencycomponents than the one or more second spatial frequency components.

In some embodiments, the first 3D image feature is closer to a viewer ofthe 3D image than the second 3D image feature; and the one or more firstspatial frequency components comprise fewer high spatial frequencycomponents than the one or more second spatial frequency components.

In some embodiments, the image processing system (600) may be furtherconfigured to perform: overemphasizing one or more high spatialfrequency components in the one or more noise patterns to a 3D imagefeature in the 3D image, wherein the 3D image feature is closer to aviewer than other 3D image features in the 3D image.

In some embodiments, the image processing system (600) may be furtherconfigured to perform: analyzing the LE and RE images; and computing thedepth information based on results of analyzing the LE and RE images.

In some embodiments, the image processing system (600) may be furtherconfigured to perform: receiving image metadata associated with the LEand RE images; and retrieving the depth information from the imagemetadata associated with the LE and RE images.

In some embodiments, the LE and RE images comprise a stereo image paircaptured from a reality by a stereoscopic camera system.

In some embodiments, the LE and RE images comprise at least onecomputer-generated image detail.

In some embodiments, the one or more noise patterns comprises at leastone of noise patterns simulating chemical films, analogous chemicalproperties of silver, chemical film color layers; noise patternssimulating noise characteristics of a specific device; noise patternssimulating a specific texture; watermarks; noises causing an imagefeature at a certain depth to go up and down, left and right, or in andout spatially; noises altering luminance or colors of an image featureat a certain depth; noise differentially applied in dark and brightareas; noises simulating a variety of response curves, saturationproperties, noise properties, devices, and media; or noises with one ormore of a variety of different distribution patterns or probabilitydistribution functions.

In some embodiments, the image processing system (600) may be furtherconfigured to perform: transforming corresponding LE and RE pixels inthe LE image with one or more transfer functions representing the one ormore noise patterns.

In some embodiments, the image processing system (600) may be furtherconfigured to perform: applying the one or more noise patterns to aplurality of image features in the 3D images at respective depths of theimage features.

In some embodiments, the image processing system (600) may be furtherconfigured to perform: removing at least a portion of embedded noises inthe LE and RE images.

In some embodiments, the one or more noise patterns are applied to theLE and RE images with at least one portion of embedded noises in the LEand RE images remaining in the LE and RE images.

In some embodiments, the image processing system (600) may be furtherconfigured to perform: tracking a viewer's geometric relationshipsrelative to an image rendering screen on which the LE and RE images areto be rendered; transforming the LE and RE images based on the viewer'sgeometric relationships relative to the image rendering screen into atransformed LE image and a transformed RE image; and rendering thetransformed LE and RE images on the image rendering screen.

In some embodiments, the image processing system (600) may be furtherconfigured to perform: converting one or more input 3D imagesrepresented, received, transmitted, or stored with one or more inputvideo signals into one or more output 3D images represented, received,transmitted, or stored with one or more output video signals.

In some embodiments, at least one of the LE and RE images comprisesimage data encoded in one of: a high dynamic range (HDR) image format, aRGB color space associated with the Academy Color Encoding Specification(ACES) standard of the Academy of Motion Picture Arts and Sciences(AMPAS), a P3 color space standard of the Digital Cinema Initiative, aReference Input Medium Metric/Reference Output Medium Metric (RIMM/ROMM)standard, an sRGB color space, a RGB color space, or a YCbCr colorspace.

In some embodiments, the image processing system (600) may be furtherconfigured to perform: rendering a noise applied LE image and a noiseapplied RE image, wherein the noise applied LE image is obtained fromapplying the one or more noise patterns to the LE image, and wherein thenoise applied RE image is obtained from applying the one or more noisepatterns to the RE image.

Embodiments include an apparatus comprising a processor and configuredto perform any one of the foregoing methods as discussed above.

Embodiments include a computer readable storage medium, comprisingsoftware instructions, which when executed by one or more processorscause performance of any one of the foregoing methods as discussedabove.

9. IMPLEMENTATION MECHANISMS—HARDWARE OVERVIEW

According to one embodiment, the techniques described herein areimplemented by one or more special-purpose computing devices. Thespecial-purpose computing devices may be hard-wired to perform thetechniques, or may include digital electronic devices such as one ormore application-specific integrated circuits (ASICs) or fieldprogrammable gate arrays (FPGAs) that are persistently programmed toperform the techniques, or may include one or more general purposehardware processors programmed to perform the techniques pursuant toprogram instructions in firmware, memory, other storage, or acombination. Such special-purpose computing devices may also combinecustom hard-wired logic, ASICs, or FPGAs with custom programming toaccomplish the techniques. The special-purpose computing devices may bedesktop computer systems, portable computer systems, handheld devices,networking devices or any other device that incorporates hard-wiredand/or program logic to implement the techniques.

For example, FIG. 8 is a block diagram that illustrates a computersystem 800 upon which an embodiment of the invention may be implemented.Computer system 800 includes a bus 802 or other communication mechanismfor communicating information, and a hardware processor 804 coupled withbus 802 for processing information. Hardware processor 804 may be, forexample, a general purpose microprocessor.

Computer system 800 also includes a main memory 806, such as a randomaccess memory (RAM) or other dynamic storage device, coupled to bus 802for storing information and instructions to be executed by processor804. Main memory 806 also may be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 804. Such instructions, when stored in storagemedia accessible to processor 804, render computer system 800 into aspecial-purpose machine that is customized to perform the operationsspecified in the instructions.

Computer system 800 further includes a read only memory (ROM) 808 orother static storage device coupled to bus 802 for storing staticinformation and instructions for processor 804. A storage device 810,such as a magnetic disk or optical disk, is provided and coupled to bus802 for storing information and instructions.

Computer system 800 may be coupled via bus 802 to a display 812, such asa liquid crystal display (LCD), for displaying information to a computeruser. An input device 814, including alphanumeric and other keys, iscoupled to bus 802 for communicating information and command selectionsto processor 804. Another type of user input device is cursor control816, such as a mouse, a trackball, or cursor direction keys forcommunicating direction information and command selections to processor804 and for controlling cursor movement on display 812. This inputdevice typically has two degrees of freedom in two axes, a first axis(e.g., x) and a second axis (e.g., y), that allows the device to specifypositions in a plane.

Computer system 800 may implement the techniques described herein usingcustomized hard-wired logic, one or more ASICs or FPGAs, firmware and/orprogram logic which in combination with the computer system causes orprograms computer system 800 to be a special-purpose machine. Accordingto one embodiment, the techniques herein are performed by computersystem 800 in response to processor 804 executing one or more sequencesof one or more instructions contained in main memory 806. Suchinstructions may be read into main memory 806 from another storagemedium, such as storage device 810. Execution of the sequences ofinstructions contained in main memory 806 causes processor 804 toperform the process steps described herein. In alternative embodiments,hard-wired circuitry may be used in place of or in combination withsoftware instructions.

The term “storage media” as used herein refers to any media that storedata and/or instructions that cause a machine to operation in a specificfashion. Such storage media may comprise non-volatile media and/orvolatile media. Non-volatile media includes, for example, optical ormagnetic disks, such as storage device 810. Volatile media includesdynamic memory, such as main memory 806. Common forms of storage mediainclude, for example, a floppy disk, a flexible disk, hard disk, solidstate drive, magnetic tape, or any other magnetic data storage medium, aCD-ROM, any other optical data storage medium, any physical medium withpatterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, anyother memory chip or cartridge.

Storage media is distinct from but may be used in conjunction withtransmission media. Transmission media participates in transferringinformation between storage media. For example, transmission mediaincludes coaxial cables, copper wire and fiber optics, including thewires that comprise bus 802. Transmission media can also take the formof acoustic or light waves, such as those generated during radio-waveand infra-red data communications.

Various forms of media may be involved in carrying one or more sequencesof one or more instructions to processor 804 for execution. For example,the instructions may initially be carried on a magnetic disk or solidstate drive of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 800 canreceive the data on the telephone line and use an infra-red transmitterto convert the data to an infra-red signal. An infra-red detector canreceive the data carried in the infra-red signal and appropriatecircuitry can place the data on bus 802. Bus 802 carries the data tomain memory 806, from which processor 804 retrieves and executes theinstructions. The instructions received by main memory 806 mayoptionally be stored on storage device 810 either before or afterexecution by processor 804.

Computer system 800 also includes a communication interface 818 coupledto bus 802. Communication interface 818 provides a two-way datacommunication coupling to a network link 820 that is connected to alocal network 822. For example, communication interface 818 may be anintegrated services digital network (ISDN) card, cable modem, satellitemodem, or a modem to provide a data communication connection to acorresponding type of telephone line. As another example, communicationinterface 818 may be a local area network (LAN) card to provide a datacommunication connection to a compatible LAN. Wireless links may also beimplemented. In any such implementation, communication interface 818sends and receives electrical, electromagnetic or optical signals thatcarry digital data streams representing various types of information.

Network link 820 typically provides data communication through one ormore networks to other data devices. For example, network link 820 mayprovide a connection through local network 822 to a host computer 824 orto data equipment operated by an Internet Service Provider (ISP) 826.ISP 826 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the“Internet” 828. Local network 822 and Internet 828 both use electrical,electromagnetic or optical signals that carry digital data streams. Thesignals through the various networks and the signals on network link 820and through communication interface 818, which carry the digital data toand from computer system 800, are example forms of transmission media.

Computer system 800 can send messages and receive data, includingprogram code, through the network(s), network link 820 and communicationinterface 818. In the Internet example, a server 830 might transmit arequested code for an application program through Internet 828, ISP 826,local network 822 and communication interface 818. The received code maybe executed by processor 804 as it is received, and/or stored in storagedevice 810, or other non-volatile storage for later execution.

10. EQUIVALENTS, EXTENSIONS, ALTERNATIVES AND MISCELLANEOUS

To illustrate a clear example, images taken from a reality are used toillustrate some aspects of the present invention. It should be notedthat other types of images may also be used in embodiments of thepresent invention. For example, images may be composite frames from twoor more different image sources. Furthermore, a part, or a whole, of animage may be sourced from a 2D image, while another part on the sameimage may be sourced from a 3D or multi-view image. Techniques asdescribed herein may be provided for these other types of images inembodiments of the present invention. To illustrate a clear example,stereoscopic images comprising LE images and RE images are used toillustrate some aspects of the present invention. It should be notedthat multi-view images may also be used in embodiments of the presentinvention. For example, two images with two different views from amulti-view image that comprises two, three, . . . , 24, or more viewsmay be used in place of a LE image and a RE image in a stereo image.Techniques as described herein may be used to process stereoscopicimages as well as multi-view images with more than two views inembodiments of the present invention. In certain embodiments of thepresent invention, multi-view images can be presented withautostereoscopic reproduction to avoid the use of 3D glasses or otherheadgear.

In the foregoing specification, possible embodiments of the inventionhave been described with reference to numerous specific details that mayvary from implementation to implementation. Thus, the sole and exclusiveindicator of what is the invention, and is intended by the applicants tobe the invention, is the set of claims that issue from this application,in the specific form in which such claims issue, including anysubsequent correction. Any definitions expressly set forth herein forterms contained in such claims shall govern the meaning of such terms asused in the claims. Hence, no limitation, element, property, feature,advantage or attribute that is not expressly recited in a claim shouldlimit the scope of such claim in any way. The specification and drawingsare, accordingly, to be regarded in an illustrative rather than arestrictive sense.

What is claimed is:
 1. A method comprising: accessing a left eye (LE)image and a right eye (RE) image that represent a 3D image; filteringthe LE image and the RE image to reduce undesirable noise; determining,based on depth information relating to the LE and RE image, one or morefirst depths of a first 3D image feature in the 3D image; and applying,after the filtering, one or more noise patterns to the first 3D imagefeature in the 3D image based on the one or more first depths of thefirst 3D image feature in the 3D image.
 2. The method of claim 1,further comprising: determining, based on the depth information relatingto the LE and RE image, one or more second depths of a second 3D imagefeature in the 3D image; and applying the one or more noise patterns tothe second 3D image feature in the 3D image based on the one or moresecond depths of the second 3D image feature in the 3D image; whereinthe one or more noise patterns are applied to the first 3D image featurewith one or more first spatial frequency components that are differentfrom one or more second spatial frequency components with which the oneor more noise patterns are applied to the second 3D image feature. 3.The method of claim 2, wherein the first 3D image feature is closer to aviewer of the 3D image than the second 3D image feature, and wherein theone or more first spatial frequency components comprise more highspatial frequency components than the one or more second spatialfrequency components.
 4. The method of claim 2, wherein the first 3Dimage feature is closer to a viewer of the 3D image than the second 3Dimage feature, and wherein the one or more first spatial frequencycomponents comprise fewer high spatial frequency components than the oneor more second spatial frequency components.
 5. The method of claim 1,further comprising: overemphasizing one or more high spatial frequencycomponents in the one or more noise patterns to a 3D image feature inthe 3D image, wherein the 3D image feature is closer to a viewer thanother 3D image features in the 3D image.
 6. The method of claim 1,further comprising: analyzing the LE and RE images; and computing thedepth information based on results of analyzing the LE and RE images. 7.The method of claim 1, further comprising: receiving image metadataassociated with the LE and RE images; and retrieving the depthinformation from the image metadata associated with the LE and REimages.
 8. The method of claim 1, wherein the LE and RE images comprisea stereo image pair captured from a reality by a stereoscopic camerasystem.
 9. The method of claim 1, wherein the LE and RE images compriseat least one computer-generated image detail.
 10. The method of claim 1,wherein the one or more noise patterns comprises at least one of (a)noise patterns simulating chemical films, analogous chemical propertiesof silver, chemical film color layers, (b) noise patterns simulatingnoise characteristics of a specific device; noise patterns simulating aspecific texture, (c) watermarks, (d) noises causing an image feature ata certain depth to go up and down, left and right, or in and outspatially, (e) noises altering luminance or colors of an image featureat a certain depth, (f) noise differentially applied in dark and brightareas; noises simulating a variety of response curves, saturationproperties, noise properties, devices, and media, or (g) noises with oneor more of a variety of different distribution patterns or probabilitydistribution functions.
 11. The method of claim 1, further comprisingapplying the one or more noise patterns to a plurality of image featuresin the 3D images at respective depths of the image features.
 12. Themethod of claim 1, further comprising removing at least a portion ofembedded noises in the LE and RE images.
 13. The method of claim 1,wherein the one or more noise patterns are applied to the LE and REimages with at least one portion of embedded noises in the LE and REimages remaining in the LE and RE images.
 14. The method of claim 1,wherein the LE image and the RE image are selected from anautostereoscopic image that comprises two or more images in differentviews.
 15. An apparatus comprising a processor and configured to performthe method recited in claim
 1. 16. A computer readable storage medium,storing software instructions, which when executed by one or moreprocessors cause performance of the method recited in claim
 1. 17. Amethod comprising: receiving a left eye (LE) image and a right eye (RE)image that represent a 3D image; determining, based on depth informationrelating to the LE and RE image, one or more first depths of a first 3Dimage feature in the 3D image; and applying one or more noise patternsto the first 3D image feature in the 3D image based on the one or morefirst depths of the first 3D image feature in the 3D image. wherein aspatial frequency of the one or more noise patterns is dependent on thedepth information.
 18. The method of claim 17, further comprising:determining, based on the depth information relating to the LE and REimage, one or more second depths of a second 3D image feature in the 3Dimage; and applying the one or more noise patterns to the second 3Dimage feature in the 3D image based on the one or more second depths ofthe second 3D image feature in the 3D image; wherein the one or morenoise patterns are applied to the first 3D image feature with one ormore first spatial frequency components that are different from one ormore second spatial frequency components with which the one or morenoise patterns are applied to the second 3D image feature.
 19. Themethod of claim 18, wherein the first 3D image feature is closer to aviewer of the 3D image than the second 3D image feature, and wherein theone or more first spatial frequency components comprise more highspatial frequency components than the one or more second spatialfrequency components.
 20. The method of claim 18, wherein the first 3Dimage feature is closer to a viewer of the 3D image than the second 3Dimage feature, and wherein the one or more first spatial frequencycomponents comprise fewer high spatial frequency components than the oneor more second spatial frequency components.
 21. The method of claim 17,further comprising: overemphasizing one or more high spatial frequencycomponents in the one or more noise patterns to a 3D image feature inthe 3D image, wherein the 3D image feature is closer to a viewer thanother 3D image features in the 3D image.
 22. The method of claim 17,wherein the LE image and the RE image are selected from anautostereoscopic image that comprises two or more images in differentviews.